Skip to content

Introduce action as indicator to indicate what one wants to do with the model #300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

JoshuaBehrens
Copy link
Contributor

Q A
Bug fix? no
New feature? zes
Docs? no
Issues Fix #114
License MIT

In other issues and pull requests the model class should be used as a simple struct, only holding scalar values and should not have any business logic effect by inheritance. This has been the case for embeddings, completions and other API calls like Whisper audio, and Dall-e images. Right now this makes it difficult to share different instances of the model class with various providers, although a lot of providers share the same models, and therefore likely the same capabilities. This eventually will reduce the amount of code you have to write to establish new providers that replicate existing API's with a well-known subset of models or add better support for custom models like OpenAI fine-tuned models or ollamas self-built models.

I see this action parameter / enum as an addition to the model. The action and model in here are like method and path in HTTP as the resource can be used in multiple different ways. Without this additional business logic effect by inheritance, it is now impossible to understand what you want to do with the model. Some models can differentiate between chat and chat completion. Right now the differentiation has been done by the model class inheritance. Some other tasks like embeddings for document indexing has been an expectation towards the model, that has been passed into the indexer. The indexer was not able to precisely use the model for an embedding. It had to trust the platform and model that it will be given an answer with an embedding result. Having the code be this unpredictable makes it difficult to maintain and work with. Same for usage of whisper and dall-e. They maybe need to get specific actions later as well.

@JoshuaBehrens JoshuaBehrens marked this pull request as ready for review August 11, 2025 03:15
@carsonbot carsonbot changed the title Introduce action as indicator to indicate what one wants to do with the model Introduce action as indicator to indicate what one wants to do with the model Aug 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Custom URL ability or generic bridges?
2 participants